Real-time Compressed-domain Spatiotemporal Video Segmentation

In this paper, a novel algorithm for the real-time, unsupervised segmentation of image sequences in the compressed domain is proposed. The algorithm utilizes the motion information present in the compressed stream in the form of P-frame forward motion vectors, as well as basic color information in the form of DC coefficients present in I-frames. An iterative rejection scheme based on the bilinear motion model is used for performing foreground/background segmentation. Further examining the temporal consistency of the output of iterative rejection, clustering to connected regions and performing region tracking, results to foreground spatiotemporal objects being formed. Background segmentation to spatiotemporal objects is also performed. Experimental results on known sequences demonstrate the efficiency of the proposed approach and reveal the potential of employing it in content-based applications such as objectbased video indexing and retrieval.